Support seal proof type switching #4873

magik6k · 2020-11-16T18:15:19Z

No description provided.

This is going to be inefficient, but it should be fine in practice.

This is terrible, but we don't (currently) update this in the miner info. Q: Maybe we should delay this by a few epochs? Some pre-commits could fail if we get a reorg.

Instead, use proof type from miner actor. This will, in turn, use the upgraded proof type if/when it's switched at runtime. TODO: Consider making this some form of config option instead?

magik6k

(looks good to me)

arajasek

Looks like it should work, and apparently it does.

Stebalien · 2020-11-17T18:26:54Z

build/params_mainnet.go

@@ -40,13 +40,10 @@ const UpgradeKumquatHeight = 170000

 // TODO: Height??
 const UpgradeCalicoHeight = 999999


^^^^^^^ We need to change this.

Stebalien · 2020-11-17T18:27:19Z

build/params_mainnet.go

@@ -40,13 +40,10 @@ const UpgradeKumquatHeight = 170000

 // TODO: Height??
 const UpgradeCalicoHeight = 999999
+const UpgradePersianHeight = UpgradeCalicoHeight + (builtin2.EpochsInDay * 2)


And decide on this.

Stebalien · 2020-11-17T18:43:46Z

go.mod

@@ -30,7 +30,7 @@ require (
 	github.com/filecoin-project/go-crypto v0.0.0-20191218222705-effae4ea9f03
 	github.com/filecoin-project/go-data-transfer v1.1.0
 	github.com/filecoin-project/go-fil-commcid v0.0.0-20200716160307-8f644712406f
-	github.com/filecoin-project/go-fil-markets v1.0.4
+	github.com/filecoin-project/go-fil-markets v1.0.5-0.20201113164554-c5eba40d5335


filecoin-project/go-fil-markets#448

Stebalien · 2020-11-17T18:44:09Z

go.mod

@@ -41,7 +41,7 @@ require (
 	github.com/filecoin-project/go-storedcounter v0.0.0-20200421200003-1c99c62e8a5b
 	github.com/filecoin-project/specs-actors v0.9.13
 	github.com/filecoin-project/specs-actors/v2 v2.3.0
-	github.com/filecoin-project/specs-storage v0.1.1-0.20200907031224-ed2e5cd13796
+	github.com/filecoin-project/specs-storage v0.1.1-0.20201105051918-5188d9774506


filecoin-project/specs-storage#14

Stebalien · 2020-11-17T18:45:19Z

node/builder.go

@@ -339,7 +340,7 @@ func Online() Option {
 			Override(new(stores.SectorIndex), From(new(*stores.Index))),
 			Override(new(dtypes.MinerID), modules.MinerID),
 			Override(new(dtypes.MinerAddress), modules.MinerAddress),
-			Override(new(*ffiwrapper.Config), modules.ProofsConfig),
+			Override(new(abi.RegisteredSealProof), modules.SealProofType),


We need to do this to fetch the params, but I'm not happy with this approach. We should, ideally, fetch the params on-demand where we actually need them. Alternatively, maybe just pre-fetch all params always?

Stebalien · 2020-11-17T18:46:02Z

node/impl/client/client.go

@@ -171,7 +165,7 @@ func (a *API) ClientStartDeal(ctx context.Context, params *api.StartDealParams)
 		EndEpoch:      calcDealExpiration(params.MinBlocksDuration, md, dealStart),
 		Price:         params.EpochPrice,
 		Collateral:    params.ProviderCollateral,
-		Rt:            rt,
+		Rt:            mi.SealProofType,


Note: we could just let the market figure out the correct seal proof type, I think.

Stebalien · 2020-11-17T18:56:58Z

node/impl/full/state.go

@@ -133,7 +138,16 @@ func (m *StateModule) StateMinerInfo(ctx context.Context, actor address.Address,
 		return miner.MinerInfo{}, xerrors.Errorf("failed to load miner actor state: %w", err)
 	}

-	return mas.Info()
+	// TODO: You know, this is terrible.


Going forward, I believe the right way to do this may be to:

Remove the proof type from the API MinerInfo type (leave it in the miner actor, just strip it from the API).

Provide a new API for getting the proof type.

Well, right-ish enough. However, probably not the highest priority.

top-org · 2020-11-24T19:22:14Z

Could anyone share to Filecoin Community about why Filecoin Team introduce new proof type at v1.2.0 ? It looks like that just wrap proof type from function into Sector.StorageRef structure ,it is better to have it but we not see any algorithm-related change at filecoin-ffi and Rust layer(Storage Proof 5.4.0) . so what is point/purpose to introduce those huge change ?

…dr-upgrade Support seal proof type switching

dkkapur · 2021-01-11T17:20:28Z

@top-org - A bug was identified in Filecoin’s Proving Subsystem which allowed miners to exploit parallelism in SDR. Specifically, the expander graph was truncated to always fall in the first 16GiB of a sector due to an integer truncation bug in the expander parents generation. The bug was reported and fixed by the rust-fil-proofs team, resulting in a new proof type being released (5.4.0). Old sectors affected cannot be added on chain since epoch 272400.

Thanks to @TyreeRoby for disclosing this issue to us and working with us to solve this in a timely manner.

Kubuxu · 2022-08-30T13:47:36Z

For future reference, the underlying fix is here (as I decided to chase it down): filecoin-project/rust-fil-proofs@0d17d74#diff-f9e7b9c546dbd93a3981eb032af33d9f52f4e7d4403192d1f26f77c886d5fa31L158-L182

magik6k and others added 25 commits November 16, 2020 19:03

Making sealing logic work with multiple seal proof types

6bea9dd

update specs-storage to fix build

d7c2c07

fix lotus seed for proof api changes

4544d21

fix storage adapter miner

219c695

load proof types in window post logic

05db5ce

This is going to be inefficient, but it should be fine in practice.

make sector-storage compile

5849072

get lotus building

9319798

Fix lotus-miner/lotus-worker build

2a7e4af

Get most tests to pass

915c5e0

update ffi

cd0cb14

Fix miners with no sectors in node tests

4d8327c

add upgrade height where SDR is required

948ee53

fix genesis

423ff06

fix calico upgrade logic

4f33c91

add basic calico test

c8ff58a

correctly set seal proof types from tests

1f91e72

Upgrade seal proof version in miner info after the upgrade.

508375a

This is terrible, but we don't (currently) update this in the miner info. Q: Maybe we should delay this by a few epochs? Some pre-commits could fail if we get a reorg.

fix compile

869867e

add a test to demonstrate SDR upgrade while committing sectors

d8afd71

Remove static proof type configuration from market subsystem

638900f

Instead, use proof type from miner actor. This will, in turn, use the upgraded proof type if/when it's switched at runtime. TODO: Consider making this some form of config option instead?

use the miner's proof type when proposing storage deals

bead8ac

update ffi

444239a

docsgen

bc05618

mod tidy

8db9c19

Fix lint warnings

e0ade92

magik6k requested review from arajasek, whyrusleeping and Kubuxu November 16, 2020 20:21

magik6k commented Nov 16, 2020

View reviewed changes

magik6k marked this pull request as ready for review November 16, 2020 20:43

arajasek added this to the 🚢Lotus v1.2.0 milestone Nov 17, 2020

arajasek approved these changes Nov 17, 2020

View reviewed changes

magik6k merged commit 693bbee into release/v1.2.0 Nov 17, 2020

magik6k deleted the feat/sdr-upgrade branch November 17, 2020 09:57

Stebalien reviewed Nov 17, 2020

View reviewed changes

bibibong pushed a commit to EpiK-Protocol/go-epik that referenced this pull request Jan 7, 2021

Merge pull request filecoin-project#4873 from filecoin-project/feat/s…

b59a49e

…dr-upgrade Support seal proof type switching

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support seal proof type switching #4873

Support seal proof type switching #4873

magik6k commented Nov 16, 2020

magik6k left a comment

arajasek left a comment

Stebalien Nov 17, 2020

Stebalien Nov 17, 2020

Stebalien Nov 17, 2020

Stebalien Nov 17, 2020

Stebalien Nov 17, 2020

Stebalien Nov 17, 2020

Stebalien Nov 17, 2020

top-org commented Nov 24, 2020

dkkapur commented Jan 11, 2021

Kubuxu commented Aug 30, 2022 •

edited

Loading

		@@ -40,13 +40,10 @@ const UpgradeKumquatHeight = 170000

		// TODO: Height??
		const UpgradeCalicoHeight = 999999

Support seal proof type switching #4873

Support seal proof type switching #4873

Conversation

magik6k commented Nov 16, 2020

magik6k left a comment

Choose a reason for hiding this comment

arajasek left a comment

Choose a reason for hiding this comment

Stebalien Nov 17, 2020

Choose a reason for hiding this comment

Stebalien Nov 17, 2020

Choose a reason for hiding this comment

Stebalien Nov 17, 2020

Choose a reason for hiding this comment

Stebalien Nov 17, 2020

Choose a reason for hiding this comment

Stebalien Nov 17, 2020

Choose a reason for hiding this comment

Stebalien Nov 17, 2020

Choose a reason for hiding this comment

Stebalien Nov 17, 2020

Choose a reason for hiding this comment

top-org commented Nov 24, 2020

dkkapur commented Jan 11, 2021

Kubuxu commented Aug 30, 2022 • edited Loading

Kubuxu commented Aug 30, 2022 •

edited

Loading